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So 
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^^M^{^^UT. ^®JUfcv^:S•fS], ^;rcv>:S-fp]^:x— 

^^M^^fc^Ur. g^i&U^cV^;^^R]. ^fcV>:;^[^:S::3.- 
50 ^-^y 7-^?fe^^®3S5^^i#i-^[if«7^— ij'fotfijd^e)^ 



(3) 



^mw-Q-e 2 8 6 1 



ft 5 (dlBfttOx^y ^-^^J^^ilo 

[If 51^:^ 8] mmm^i^^^^^:. 

^x\ mmm^&:mf^n^mii-Jk^tzL^mi:^i^(07&Bii- 
^oxi^<>m^(oi^^m^(om^y'-^ t:. m^^^^iz 

m^^5(Oy<y^'^^ti^m^ so 



m^(D^x^ mmm^&m^w^^m:d^=f^mvtzm^:^\^ 
mi(D^mi^^^x\,^^msc(o^mm^<om^y^—^ 

Iff 1 3 ] m^^^^^tK 

nm^\^^^^mMx-(Dm^m9i<omM^i^ t -s: 
;6-o. nM&m::^-mm:^m:iMi.xmm\z^m'r^ 

^^i-^r i:^#^^-ra. if^SS-l 20f5rtL36^«c: 

m:^mmzM\^xM.tL\^^:^\^^^—^ibm^'r^m^?< 
x<ommm6^\z.$itt^--^v v^mm^-^x^m'^^^^ 



1 



[00 0 1] 10 
[0 0 0 2] 

^4-2 7 5 7 8 3#^^lc:^$tL^^®:d5$>^o 

[0 0 0 3] m2, 4.},'x^(D^^(D'^m(om^mx*^^ . 
A^D^^^^. 3 3 \msmm^h^</ ^^m^^^^ 

ir^y<y ^^m^-B-^^^. 3 4 fi^^y ^-^M^copj'S 20 
X- ^ J =7 -^M^IBlt^^. 3 5 \mM^ 

[0 0 0 4] S2 5«. SJ^Mfi^ir^-^y 7-^M*coja^5r 
[0 0 0 5] ^m^ti:^ 7 3 1 ^seoSiHje^-^-ctTJ^.? 

^>o S^iil^fiA/D^^^m 3 2 J: o T-^iX-ettT^i^ so 

[0 0 0 6] 7^iii«'g^^^^3 3 fi. 

[0 0 0 7] ^2 5(c:;^i-J: 5f^:^.^.^*'C^^ 

[0 0 0 8] SJ^M«<^tte. :3&^f^«. «]^^{C;?7^ ^ 

[0 0 10] ^^^tcj^. :^ =7 <Ty^^^<'7 :^ 

^^^^ 3 5 d^x-^y 7-^®»d^e>^KS«J^JS»l®«®t- 

i-So , 50 
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[0011] 37 V— Ay 3 6f2: 

[0 0 1 2] ^^<7:>/'^y ^-^B^^M^i, :L(D^ofm 

[0 0 13] 

o ^r^^ C -5 r i: t ^ ^ V > 5 

[0 0 14] ^fc. it^iij^^— ^^-^y 

[0 0 15] ;$:^5gfijil5(^^tc:^;^, :^^^{ciott 
[0 0 16] 

[0 0 17] :$:mm<D^^^^-t^±it><om2<o^^ 

[0 0 18] ^^m(Dn^^m^ir<)tn^(om3<o^^ 
tt. ^mPs^S:M(owm^m\^^x^W)'r^.^^n.^.tv 

x^(omm^^vrcm^(om^'f—^^^n'r<>m^ 
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[0 0 19] :$:^m(ou^^^m^'rc>rzib(om4<D^^ 
[00201 ^^m<o^^^m^'r<>tii!b<om5<o^m 

[0 0 2 1] *^^(oefi<j^ia«-rsfcae)OS6(o^^ 



[0022] :^^m(D^^^m^^^f:iit^(D^7 <D^^ 

[0 0 2 3] ^^m(o ^^^m^-r^tzibcDm 8 (D^^ 

[0 0 2 4] :$:mm(On^^m^^<>it}^(Dm9(D^^ 

isM^^Mtf »^^2is^a6 fcm.^.^gt-:fe tt ^ mem^ 

^-ce t six. fc r ^Sri^« i -f- S o 

[0 0 2 5] :^mm(o^^^m^i'<)ti^<omi 0<O^ 

[0 0 2 6] :*:^^(0SW^jS^-r^)fci^)^0^1 1 (O^ 



(6) 



9-62861 



[0 0 2 7] :$:^m<D^&^^W:^'T^fz^(Dm 1 2 (D^ io 

^t^^o 20 

[0 0 2 8] :$i^m(O^^^^M^^<>fz.}h(0% 1 Z(0^ 
[0 0 2 91 ^^m(OU^^m^'t^t^^<0^ lACD^ 30 

^mt^ m^mm^^M\^xM.ti^^:^m^^—^^^^m-r 
n^^t. mi^vx\^^^m^<o^(D^M\c:^ii'^^<y^ 

[0 0 3 0] :$:^m(D^^^m^'t^tzi^<Dm 1 50^ 

^:Ltt:1^mtir:b. so 
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[0 0 3 1] :*:^§^(Dge^$:^-r-53ti6<^^l 6 

[0 0 3 2 J *^^(^g^^^-r5fc^6iOmi 7£0^ 

[0 0 3 3] :$:mm(On&^^m^'r<>t:Li^<omi S(D^ 
^1 5. 16. 1 7<7>#^Jc::feVN-C. M'teT^— 

(omm^^^J:^^. ^(D^co^:ky^—i^'^^X\:i^X(Dm 

[0 0 3 4] 

'^^^ii^^^^^^f^^-^^%(Dwm'-^m^^x^m^^ 
.^.^^Mt vx^(Dmm^w^\^tim^(om^v'-^^z 

^-ro^^v^'^^i^#^'^J5^uTi3< ^^t>E-. ^^^^ 

[0 0 3 5] ^fc. ^^iij^^msiBit^m;5^^5^-t-^m 

[0 0 3 6] -i^fzM^^^^mK immm^h^s^ 
[0 0 3 7] ^ti^ m^^m^mt^^<j ^^mmt:^m 

[0 0 3 8] ^fc. ::^t:^.^ili:^^l^o;^p< ^T-a^bfc 
[0 0 3 9] M^ISit^SdN M^r'-i?'^^^ 

<>^m-kA^^<^^:itt^x^<><DX\ mm^m^'^^ 

[0 0 4 0] 
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[0 0 4 1] mmm d e i :^^m<om—<D^^ 
[004 3] k/dm^^mt. ^mmmm: loitm 

[0 044] mm^jE^^ 1 0 3 f^. y^i^^Mt^tltc 

m^^(DmjE^n^£v. M^iBSt^ai o.4fi. « 

[0 0 4 5] y<y "y-^m^M^^m^ 1 0 5 fi. ®^IS 

[0 04 6] a^>fi!:jHf 1 0 6 ^-^ 
Ml^^u^^^^ 10 5 1^^^ v±^^M\^isn^^<y "7 20 

[0 0 4 7] ^-^M^iait^^i 0 7(i. /^y 
^ . ^MMmn^W^^^ 10 6 15 ^x.ltMMJSL 

[0 04 8] Jg;^A;^^^ 1 0 8 > S^^Olf 
[004 9] ^;^ili^fflMlElt^^ 1 0 9 ^<7) 

[0 0 5 0] mmtLm.^n^^ 1 1 o t^. ^^®^®m 

fSte^^l 0 9d5-£^^§UT\/^5m^.i^>"C^:^*<OIlf^O 
m.^.. m.^l:^fS]^om^> =^-^t-nyv^J^f^^^ \ 0 8^ 

[0 0 5 1] B'^^^^iZi^^g: 1 1 1 ti. m^^WVrn 40 ^ 

^^1 1 o±^v^n\^fzMMMm^^n<>^<j'7-7m^(o 

[00 5 2] M^-^^^^ 1 1 2 M>^^;^iZi^^^ 

1 1 \ t'^m.wb.hjfz:^<j'7^m^'f^-^^m^. M^ae 
etfS^^i 1 o^5tfStbfemiS:*r^<^M«^'&^i- 

[00 5 3] 1 1 3 tt, M^-g-^^^ 1 1 2 



[00 54] ^^J^^M 1 0 1 tt. 0 2 - a 

tC:^i-J; p:^^^>£r3^o, ^ 2 - a fc:j3V^T. 2 0 1 
WM^^ 202 1^;^; 7f ^1^. 2 0 3 f^-^^T*^ Oo 

[0 0 5 5] ^Jlt^2 0 1 fi. Ti^l^(DmM^^-olL^ 

[0 0 5 6] :^7^^*P2 0 2|i, n-^cO;^7p« 7:d^e>^^ 

[0 0 5 7] 0 1 <^M®CO#:6^bCOfi^^4 5 

m^l^. ;577« y^2 0 2C0^;J;;^ ^f*:ds^l^M2 

(clEMU 0 1 <:o|ft_b}cfeo— o(7:)m.^.^^e>^ 

[0058] ^2-bfi. ±mm^mm,\ o i r-jg^u 

[00 59] 20 4 2 0 5 t:i#:*r rp](O^M^ 

X^^o aj^®^2 0 5tt. ^1.^.2 0 4^*'L"i:r^n 

[0 0 6 0] ^:^'7U2 0 2(D^X(D:^;^'7<Omm^y^ 

0 3 ^^VvX:^Mi^SSl 0 1 ^^ii^-^/^dS^S^ 

[00 6 1] S3fi. m^m^t^<j^'^m^t(D^m: 

IS^^^LfcIS-e^-5o !33t-:fcV>T. 3 0 1 tiS^M 
^. 3 0 2«/^y ^-^M^. 3 0 Sti^M^^^:^^ 

[0 0 6 2] m.^^1=^^^-r5li:35M 

[0 0 6 3] 'i5^xy^^(D^.^.<Dmm(D^^m^ 

:Lt\zx^. ^^r^wi^<omm:^\^. mMm.m^^^x 
i^5<o -(om. Wt^m<Di^mmemmii 0 i<omn:^m 

3 0 id^e^^y 7-^liilfe3 0 2±---, W.M^'P^bt'f'^ 
SmSJ^^ff*5o rtttcj: t)/^y7^M«3 0 2i:«^ 
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[0 0 6 4] Z(OX0\Zl.X. -0<D«;^JC^5V^T^ 
[0 0 6 5] ^Ji^Sj^^g 1 0 1 ^^Wb^^t^y!)- 

bn^^(Dx\ m^m^(ou.^.hmm\^x^W)\^. 

[0 0 6 6] m.^.^MtS ^^^^^ 1 0 6 (i. r.<7>«W 

[0 0 6 7] ®^?f^^co^Sfco^/^rPL.<i& 

[0 0 6 8] (:^T^yys\-l)m7r^mm\^Mir:5^—'f<D 
[0 0 6 9] {:^'ry':^s\-2)^m/^—y^(omt^^.^.X 

m^umf^n^^ 1 1 0 mTr^m^mmmm,^^ i o 

m^^X7<.t}Vfc^M^^£^m:^\^tt^^. 
#^10 9tc:gS-rOo 

[0 0 7 0] (:^y'iyy^SU3):^'riy':fSl-2Xm^&:m.^ 
n^^l 1 0d5^:^|^7-^^^^^^g{;::.^j:j;;^^^<y ^-^M 

•^li'felBtg^^i 0 7:d>e>Sc;^ttJi-o 

[0 0 7 1 ] (^^:y:7'Sl-4)M^-^^^^l 1 2 M 

[0 0 7 2] i:^y'^yys\~5)m^'^^^mil 2i>^^^ 
[0 0 7 3] (;^7^5'^Sl-6)^-ifd^ ^^^^113 

[0 0 7 4] (;^^5/7^Sl-7)St/;^7":y >^Sl-HC*£!i3g^^ 



(8) 4^§i^ 9-6 2 8 6 1 

14 

[0 0 7 5] ;^7^s^:/si-6fc:fet^T. ^— ifd^Jg^A;^ 
^^1 0 8{iiJ:oT?g^-C#C):^Sfi. mm. ^51, ^ 
Jh, :&lHlfe. iEJ^m. ±^m. T0te> ^/h. ^ 

(^>3otc:ov^Tf±^ •t<^:>3o<?5V^i"i^^=*<75:^g3^— lUJi 

10 m&'t^^tx^ nM.^W3twM^\^(om'^:^mm^n 
Jg;^'^tg^-roo JbT;^rp]oiHi^. it^. ^ 

[0 0 7 6] rn^c^^^f^i^St-^'r^. :^y'^yzfS\ 

20 t) . ^nM^<r>^<y '^'^mm^n.^Mmm-^t ur 1 - 
A <i>. A f t-rso 

[00 7 7] (a)tu3l:«.^.{i®^— oiii6-5o "T^c^t? 

J^.-^.*-!^. ■t-?t^t>*>»^^oat^T:fe■^R)^c^ur9o^J: 19 

[0 0 7 8] (b)^il:m,§&g^-oMU. ifcCO^^ili 

i:>'h^^w<omn:^m^^Mi.x9o^x^i^%\i^mxhti 
[0 0 7 9] (c)^»jh:m^.ffie. ^^m. MMmMt 

[0 0 8 0] (d)*rlHlte:«,^firM. m.^.^Mlt. 

xo:>m^m^(o^^ t-r^o fe^rcu i-etirtfrit. mil 

-C(7)miS:fe-|R3{j:^U*e::^SA 0 ffjff ^U)t(e - A 
50 [00.81] ie)iol^mMMUm. 
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[0 0 8 2] (f)±iHite:m,^ffia. m^mmfit. w^j& 
[0 0 8 3] (g)Tm^ 'MM&im. MMm-m^-t. 

^^V^-fi^^^;^^^i^^ttrv^^)t^^^O^J;l^i. (a)(b)0 
[0 0 8 4] (h)i!::k'MM&m. ^M^m-i. ^^.^X 

-r-So 20 

[0 0 8 5] (i)®/h:«.^.fi:g. ^=^;^fR]f5. m^^Mx- 

[0 0 8 6]»«ctc. :i(OX^\^l^x^&\^tcmM&WL. 
mAmm^^Mvx^ mm^^^^i i 2«. ■ 

mm(Dm^^^'f'<>o :i^L.x^^i.tim^^. 30 
[0 0 8 7] si±<D^o\^^mMm\^^in}t. ±m<ox 

<^>T% f^(Dmmm^Wi^^z t:A^x^ 

[0 08 8] /^jB. 2^^J£'^j-eii^^y ^•^M'&^m.^.^ 
^^L^tir^^rnxm^^l^tcibK n.^.^^^t-t'r<>F^mm AO 

%:m^^xhmm(Di^^7bmbtt^. 

[0 08 9] (HJS^J 2 ) S 5 fi. 5*:^§g<Z>S-0^^ 
[0 0 9 0] ^5$C*3V^"C. ^^^g5 0 Ifi. 

[0 0 9 1] A/D^^^^ 1 0 2 (i. 50 so 
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[00 9 2] m^^lE^^ 1 0 3 tt. y'i^^Mt^ fx^tL 
[0 0 9 3] M'^IElt^^ 1 0 4 J^. m'^E-^fltzM^^ 

m^^ms 0 ii65[^-m,^.{-:fcv>r^ufe^^o^#j 
[00 94] m.^.fifiif 1 0 6 fi. 

H^a^J'S'^^^ 5 0 5 dS-g-J5^bfc€^,^t2:*3(tS^</ 7 

[00 9 5] ^"^ii«i5«^^ 1 0 7 mmi^tz. 

[0 0 9 6] ^it^^a 5 0 1 s ^J;t^f . ^ 6 - a 
jC^-r J: 5 fm^^n^. ^ 6 - a |r:f3V>r. 6 0 1 
;?7p<^. 6 0 2 mte^J^. 6 0 3ti'^*:'T?^6o :^ ^ 
^601 [Hlte-^M 6 0 2 (dlfe t) #tt ^3 n-So mte-^M 
6 0 2 ti, iij^'y^OX <r>'^:^'7^*l>. '^f^t>%^^m 
mc^-t^^.^MW^'^'t-h\^X^ ^/vh(7>{El^ 

[0097] S6-bfi. ^SJ^^® 5 0 1 r-SJ^U 

^VN-C. 6 0 4 fitS,fe. 6 0 5 \%^^^(O^M'W^X^ 
n;^fp3(d5^#JUrJt^bfcJ^'^. ®^jll^6 0 5fi 

m^6 0 A^^*L-}L't^Ti^'^(D^'^^(omm-^T^^ir 

-So 

[0 0 9 8] — 000^.^.(0^ a ^;j7p« ^^m^^-^-Tii 

j^bfcm. ^^6 0 3^^VNT^/h^^fctt^®)L. 

[ 0 0 9 9 ] ^-^y y-^^^M-^^^^^ 0 5 f^. JblEc^ 
J: 5 UTSj^^ttfcMlfelEtS^^ 10 4 is^W^-r^m 
^=f-^^m\*^. ^^^^5 0 l^^^Ufc^iX^ 

[0 10 0] -^ii^7^.^:a-f^%^^UT/^ 

[0101] T-^pj^JIJ-g^^ft^^ 5 0 5 $ 
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[0 10 2] :i(DXo\^\^X. ^^M^(Oy<y'7^m^ 
[0103] "^±.(0^ 5 f-^^ife-^Jtw J;tLl^s 

[0 10 4] :^^:lfe'B?ijT*tm^t-^^Si^lM'^ 

[0105] m^rn 3 ) ^ 7 20 
[0 10 6] ^a7^rioV^T. M^l5lt#^l 0 4|i. 5g 
[ 0 1 0 7 ] ^^M'fe^'J'^^^^ 1 0 5 t^. U^fE 

ti#^ 10 4 ^s^j^-r o lii^T^- 1^ e> . f^-m.^. 

[0108] m.'^.'fugit*^^^^ 1 0 6 ti. ^^y ^ ^ 30 

[0 10 9] /^y ^-^®>fe$^fij^^7 0 1 /-^y ^-^ 

[0 110] :$^t>J/^y ^■^IiJ«aBlS^^7 o 2f±, -/-^y 
^•^M^^^-Sd^^jds^^JUfc^N-y ^-^iU^oM^^T^-^ 

[0111] ^;^iif>^®Mf5^^^ 1 0 9 Ji. ^— if 60 
[0112] m^^W^m^^WL 7 0 3 ti. ^^M^M 

[0113] m^nWh^^^ 7 0 4 M^fiH^ffl 50 
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^^7 0 3:6s^igLfcm.^ffi:M. a^^S-fPliOgs^aJ^o 
TV>c>®«$:. i^^^<J ^^IillfefE'|t^^7 0 2 3&^P>Sg 

[01 14] fi^A;tr^^ 1 0 8 . e^^^W^Npfr. 

;^fp] A;^t* o 1j -5 o 

[0115] M^&Mft-^^^ 1 1 0 ti. m.7mmmm 

lElt^^l 0 9>i5^^^UTV>^m^,#.-C^^cf>Oil|^(Z> 

m.M.. W^':^n<om.t^ 1 0 8 ^ 

[0116] m%.%^-^^ 7 0 5 Ji. M-i^^^^Z.^^^ 

7 0 4 ^dSic^iZ.Ayfcr^'fd^^y ^ Ij 
Ife&Mff^^^i 1 Odstf^Ufcm^:;^fRi<7)iii«^'g'^ 

[0117] 1 1 3 (i. M^-a^^^^ 7 0 5 

[ 0 1 1 8 ] !3 8 J^. ^^y ^^M^^^-fd^^ 7 0 1 :65^T 
9 ilill^:9-fiJ<7:>m:^llx-&^o IS 8 ^;Ii5v^T. 8 o i 
vN-y 8 0 2;5S5^fyiij«fcT-&^o 

[ 0 1 1 9 ] /^y ^-^M'^^y-e^^^^ 10 5 ;65'^^ufc 

:^Ir3^7c X (2 X i - l)/'k [rad]. 8 ;^[R](Z>f--^ 

9tth fc^e-^k^y ^-^iu^KiBtt^^ 7 0 2 
[0 12 0] ^^fc. m^w±^(Dmm\^-:>\^^x. pu< 

X. ^(D^m^mmir^, 

[0 12 1] (^y'^y':fS3-l)m^<Dmi^mM.^mM 

fig. ^Xx^mn:^\Ri^mi^:^\^kVX. ^(0^]^ib^^ 

—^^^ibm^^^^m 7 0 5 t^m^mi^^'^^ 

¥^1 1 3:&s^^'r5o 

[0 12 2] (^i^^r yZ^S 3-2):=^— ^(DT^tl^^i:^ etc 

[0 12 3] {:^'riyyS3-3)^—'^^^, m^^W^l 
1 3i)m^-r^m^t:^x. Jg^A;^^^l O 8^>^ 

[0 12 4] (;^7"5/>^S 3-4)M^figtf^^^l 1 

0:6^ mTT^mmmmm^^mi o 9^5^^tb-cv^om^ 



(11) 
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[0 12 5] (:^y'^y:fS3-5)m^^m=f-m^^7 0 

[0 12 6] (>^7":y>^S 3-6)tii^Sc;^72.^#^7 0 
[0 12 7] (:J^7"s/:7'S 3- 7)®^-^J5fe^^7 0 5 

[01281 (y^y" 5/ IT'S 3 - B)m^^^^^ 7 OSt^ 20 

[0129] (;^Tiyy^S 3-9)Wn:^y'iy':fS 3- 2 
l^i^^^Mb. :^T'5/:^S 3-3;6^^;:^7^-/:/S 3-8 

(1) ^it : mMMW: : n. ^m;;^fp] : 



^S^o^T^^g:^i-i^^^. >^$rfelt. ^i^(o^m 

[0 1 3 0] ;^^>y:/s 3-5. 6. 7{j:iott'5*Df^tc 

^7 0 3}JS]jfe{c^:^-r^lif«0«.fefi:e. «)S^fRl. 

^CT% mmMxm^^(om^^ I (t) . — o 

I (t-1) . l^(0^.^xm 

T^-r^m^^i (t-hi) ^i-oo ^ft. I (t) (Dffi 
[0131] 010 \z.^'rmM}(D—^<omt br. i • 

( t - 1 ) tC:}bNtS.«,SffiM##dSii - 1 

[0132] ^-r. I ( t - 1 ) ^ I ( t ) xu.^Mm 

[0 13 3] bTtdSoT. ^ffli^tt'Saf^^. ^<Dm(r> 

(0 t . <f» t) 



(2) Sfr5i : : B + 1 . m^;^ipj : ( 0 1 



(3) mm-\-&^^ 
(4) 

(5) ttrit-fJiiE]^ 

(6) Mii+Tme 



[0134] m^m^T^^^^m 7 0 4 r 5 uT^s 
[0135] m^^^^mi 0 5 fi. B^&Mtf^#^ 

1 0 9:65^^Ufcm.^.teS. m^^lR3{;i^UT. -^rCO^ 

(C>(S«6^/cf M^m^^^-r^o M^^^aZ.^^^ 7 0 4 
:65^^a2.A/faiI^7^-i5'(7>'fT% iS^fiMtf 1 0 

ij'^m^^^b. ^fih(om^y'-^ ^^mm^m±\^ 
[0 13 6] u±(oxo\z.^%-mm\^^f\^\^. y<y^-^ 

[0 13 7] 9'foaf^^:^(;iJ:5^:^M«^ 



n+ 1 
n + 1 
n + 1 
n+ 1 

30 



4> t) 

(9 t H- A 0^ 4> t) 
( 9 t — A 0 , <|> t ) 
( 9 t , ^ t 4- A <f>) 
(9 t. <^ t - A <^^) 



ti:m^y'-i^ ^hht^\:^i^m^i2^AyX^< :i t:i)^Xt ^ 
[0138] :^|l:S^J{^::fcv^r^i>''^y ^-^li^^^ 

[0 13 9] *^^S^J^;l:fev^rfl. ^©^RtDz-^y 
[0 14 0] ^fc. :2^^:te^J{^::fev^■c^l. ^my<y 

•miSi^m\^^xm^m^^^^i'^m^i^:is}^^x. 
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[0141] (mmm^) mii^. :^^m(om^(o^ 

[0142] ^11 ^C:fc^.^T. Hj^IStt^^ 1 0 4 f^. 

[0143] ^M&mmmn^^^ io6\^. mmmm 
[0144] m^m^^mm^^^ 1 1 o 1 mms. 
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(54) PANORAMIC VIDEO DEVICE 

(57)Abstract 

PROBLEM TO BE SOLVED: To display a panoramic 
video where viewpoint/glance direction are continuously 
changed in accordance with the designation of a user. 
SOLUTION: A panoramic picture synthesis means 105 
synthesizes the panoramic pictures for respective 
viewpoints based on picture data of a picture which a 
picture storage means 1 04 holds and which is obtained 
by photographing the periphery of a point moving in a 
space by drawing an arbitrary track with the point as the 
viewpoint. A viewpoint position information setting 
means 1 06 sets continuous viewpoint position 
information in the respective panoramic pictures. At the 
time of reproduction, a picture position calculation 
means 110 calculates the position in the space of a 
display picture and a picture synthesis means 1 1 2 
synthesizes the display pictures from information on the 
viewpoint position and the glance direction of the display 
picture which a display picture history storage means 
1 09 holds and from information of the viewpoint/glance 
direction which are inputted by an indication input means 108 by the user. 
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CLAIMS 



[Claim(s)] 

[Claim 1] An image storage means to hold the image data of the image which photoed the 
perimeter by making into a view the point which draws the locus of arbitration and moves in the 
inside of space, A panorama image-sequences composition means to compound a panorama 
image for every view based on the image data which said image storage means holds, A view 
positional information setting means to set view positional information as the panorama image 
which said panorama image-sequences composition means compounded. Panorama image edit 
equipment characterized by providing a panorama image storage means to hold the panorama 
image data of the panorama image which said panorama image-sequences composition means 
compounded with the view positional information which said view positional information setting 
means set up. 

[Claim 2] The panorama image edit equipment according to claim 1 characterized by to have 
replaced the image data of the division image with which the panorama image which a panorama 
image-sequences composition means compounded divided into two or more images, a panorama 
image division means set the space position relation centering on a view as each division image 
provided, and said panorama image division means created a panorama image storage means with 
a division panorama image storage means hold with the view positional, information which a view 
positional-information setting means set up. 

[Claim 3] An image storage means to hold the image data of the image which photoed the 
perimeter by making into a view the point which draws the locus of arbitration and moves in the 
inside of space, A view positional information setting means to set up view positional information - 
to the image data which said image storage means holds, A photography image location means to 
set up the image positional information in the space centering on a view to the image data which 
said image storage means holds. Panorama image edit equipment characterized by providing a 
positional information storage means to hold the view positional information which said view 
positional information setting means set up, and the image positional information which said 
photography image location means set up. 

[Claim 4] Level and an image storage means to hold the image data of the image with which both 
vertical field angles made it move to arbitration, and photoed 1 80 degrees or more of a certain 
cameras in space, An image division means to divide the image for every photography time of 
day into two or more images to the image data which said image storage means holds, A view 
positional information setting means to set up view positional information for every photography 
time of day to the division image which said image division means makes, A division image 
location means to set up the image positional information in the space centering on a view to the 
division image which said image division means makes. Panorama image edit equipment 
characterized by providing a division photography image storage means to hold the division image 
which said image division means makes with the view positional information which said view 
positional information setting means sets up, and the image positional information which said 
photography image location means sets up. 

[Claim 5] A display image hysteresis storage means to hold the view location of a display image, 
and the hysteresis of the direction of a look, A directions input means by which a user shows .a 
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direction to move and a direction to see to a display image, The view location of the display 
image which said display image hysteresis storage means holds, the view location of the image 
displayed on a degree from the amount of directions of the direction of a look, and said 
directions input means, and an image location count means to calculate the direction of a look. 
An image reading means to read the panorama image data in the view location for which said 
image location count means asked out of the panorama image data which panorama image edit 
equipment according to claim 1 holds. An image composition means to compound the display 
image of the direction of a look for which said image location count means asked from the 
panorama image data which said image reading means read, Panorama image equipment 
characterized by providing a display means to display the image which said image composition 
means compounded, and carrying out by repeating the input of the amount of directions by the 
user, composition of the image to it, and a display [claim 6] A display image hysteresis storage 
means to hold the view location of a display image, and the hysteresis of the direction of a look. 
The view location which the user will direct next based on the hysteresis of the view location 
which said positional information storage means holds, and the direction of a look, and an image 
location anticipation means to expect two or more sets of candidates of the direction of a look. 
An image reading means to read the panorama image data in two or more view location 
candidates whom said image location anticipation means expected out of the panorama image 
data which panorama image edit equipment according to claim 1 holds, A directions input means 
by which a user shows a direction to move and a direction to see to a display image. The view 
location of the display image which said display image hysteresis storage means holds, the view 
location of the image displayed on a degree from the direction of a look, and the contents of 
directions of said directions input means, and an image location count means to calculate the 
direction of a look. An image composition means to compound the display image of a view 
location and the direction of a look for which said image location count means asked from the 
panorama image data which said image reading means read. Panorama image equipment 
characterized by providing a display means to display the image which said image composition 
means compounded, and carrying out by repeating anticipation of the input of the amount of 
directions by the user, composition of the image to it, a display, and the image location of degree 
image. 

[Claim 7] An image reading means in the division image in the view location for which the image 
location count means asked The image data of two or more images located near the direction of 
a look which said image location count means searched for It replaces with the image reading 
means read out of the image data which panorama image edit equipment according to claim 2 
holds. Panorama image equipment according to claim 5 characterized by replacing with an image 
composition means to compound the display image of the direction of a look for which said image 
location count means asked from two or more image data into which said image reading means 
read the image composition means. 

[Claim 8] An image reading means in the division image in the view location candidate whom the 
image location anticipation means expected The image data of two or more images located near 
[ which said image location anticipation means expected ] the direction candidate of a look It 
replaces with the image reading means read out of the image data which panorama image edit 
equipment according to claim 2 holds. The view location where said image location count means 
searched for the image composition means from two or more image data which said image 
reading means read, panorama image equipment according to claim 6 characterized by replacing 
with an image composition means to compound the display image of the direction of a look. 
[Claim 9] An image reading means in the photography image in the view location for which the 
image location count means asked The image data of two or more images which took a 
photograph near the direction of a look which said image location count means searched for It 
replaces with the image reading means read out of the image data which panorama image edit 
equipment according to claim 3 holds. Panorama image equipment according to claim 5 
characterized by replacing with an image composition means to compound the display image of 
the direction of a look for which said image location count means asked from two or more image 
data into which said image reading means read the image composition means. 
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[Claim 1 0] An image reading means in the photography image in the view location candidate 
whom the image location anticipation means expected The image data of two or more images 
which took a photograph near [ which said image location anticipation means expected ] the 
direction candidate of a look It replaces with the image reading means read out of the image data 
which panorama image edit equipment according to claim 3 holds. The view location where said 
image location count means searched for the image composition means from two or more image 
data which said image reading means read, panorama image equipment according to claim 6 
characterized by replacing with an image composition means to compound the display image of 
the direction of a look. 

[Claim 1 1] An image reading means in the division image in the view location for which the image 
location count means asked The image data of two or more division images with which it is 
reflected near the direction of a look which said image location count means searched for It 
replaces with the image reading means read out of the image data which panorama image edit 
equipment according to claim 4 holds. Panorama image equipment of claim 5 characterized by 
replacing with an image composition means to compound the display image of the direction of a 
look for which said image location count means asked from two or more image data into which 
said image reading means read the image composition means. 

[Claim 12] An image reading means in the division image in the view location candidate whom the 
image location anticipation means expected The image data of two or more division images with 
which it is reflected near [ which said image location anticipation means expected ] the direction 
candidate of a look It replaces with the image reading means read out of the image data which 
panorama image edit equipment according to claim 4 holds. The view location where said image 
location count means searched for the image composition means from two or more image data 
which said image reading means read, panorama image equipment according to claim 6 
characterized by replacing with an image composition means to compound the display image of 
the direction of a look. 

[Claim 1 3] In the view location and the direction of a look which the image location count means 
searched for [ the image composition means ] When the direction of a look is specified that a 
view location moves perpendicularly to the direction of a look in accordance with the direction of 
a look of the display image in this time it is on the same flat surface as the display image in this 
time, and is the same as the movement magnitude of a view in an image core — panorama image 
equipment given in any of claims 5-12 which are characterized by compounding the image in the 
location made to ****** they are. 

[Claim 14] A panorama image storage means to hold the time series panorama image which 
carried out fixed time amount photography of the image of the perimeter in one view, A display 
image hysteresis storage means to hold the hysteresis of the direction of a look of a display 
image, and a directions input means by which a user shows a direction to see to a display image. 
An image location count means to calculate the direction of a look of the display image which 
said display image hysteresis storage means holds, and the direction of a look of the image 
displayed on a degree from the contents of directions of said directions input means. An image 
reading means to read the image data of the panorama image in the next time of day of the 
image currently displayed from, said panorama image storage. An image composition means to 
compound the display image of the direction of a look for which said image location count means 
asked from the panorama image data which said image reading means read, Panorama image 
equipment characterized by providing a display means to display the image which said image 
composition means compounded, and carrying out by repeating composition of the directions 
input by the user, and the image corresponding to it, and a display. 

[Claim 1 5] Image storage characterized by arranging the image data of the image with which a 
view location and the direction of a look continue on the recording track with which a record 
medium continues or adjoins to the perimeter image divided in the direction of plurality for every 
view. 

[Claim 16] Image storage characterized by recording the image data of the image sequences for 
every direction on a separate record medium, interlocking the head to all record media to the 
image data of an image which divided the perimeter image in two or more directions at the time 
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of read-out of image data, and making it move. 

[Claim 1 7] Image storage according to claim 1 6 characterized by recording the image data of the 
image of the same view on the same location on each record medium. 

[Claim 1 8] Image storage according to claim 1 5. 1 6. or 1 7 characterized by compressing image 
data and arranging the size of all image data with the maximum data size in it. 



[Translation done.] 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] This invention relates to edit of the panorama image which photoed the 

omnidirection. and a display. 

[0002] 

[Description of the Prior Art] Conventionally, as image equipment using the panorama image 
which photoed the perimeter of one certain view, there is equipment shown, for example in JP,4- 
275783A 

[0003] Drawing 24 is the block diagram of this conventional equipment, and an A/D-conversion 
means by which 31 digitizes the camera for photography and 32 digitizes a photography image, a 
panorama image composition means by which 33 compounds a panorama image from a 
photography image, a panorama image storage means by which 34 holds the image data of a 
panorama image, a geometric conversion means by which 35 compounds a display image from a 
panorama image by geometric conversion, and 36 are the frame memories holding the image data 
of the compound image. 

[0004] Drawing 25 shows the geometry-physical relationship of a photography image and a 
panorama image, 31 is a photography image and 32 is a panorama image. 

[0005] Photography is performed by rotating a camera 31 360 degrees, and obtains two or more 
photography images which divided the perimeter of one view. A photography image is digital- 
data-ized by the A/D-conversion means 32, respectively. 

[0006] The panorama image composition means 33 considers the image side of the cylinder 
centering on a view, and projects a photography image on the image side. 
[0007] At this time, as shown in drawing 25 , the location of photography equipment and a 
direction are determined in system of coordinates centering on a view, and the perspective 
projection centering on a view is performed. 

[0008] The location of a photography image and a direction have the approach of measuring the 
control input of a camera at the time of photography, and a method of determining a mutual 
relative position in quest of correlation of the pixel value data between each photography image 
after photography. 

[0009] Thus, the cylinder image which projected the photography image of all directions is a 
panorama image, and the image data of this panorama image is once accumulated in the 
panorama image storage means 34. 

[001 0] At the time of playback, if the photography parameter (Pan theta. Tilt phi. Zoom z) of an 
imagination camera is inputted, the geometric conversion means 35 will perform transparent 
transformation centering on a view from a panorama image to an imagination photography image 
side, and will compound an image. 

[0011] The compounded image is accumulated in a frame memory 36, and is outputted to a 
display etc. 

[0012] By having such a configuration, conventional panorama image equipment could generate 
immediately the image of the direction which a user desires to the world which exists in the 
perimeter of one view, could display it, and while the user operated the camera on that spot, it 
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was able to take out the same efFectrveness as having seen the image. 
[0013] 

[Problem(s) to be Solved by the Invention] However, with the above configurations, it is only 
looking around the perimeter of one view, and what is in a perimeter had the technical problem 
that breadth of the space which exists in the perimeter of a view could not be sensed even if it 
understands. 

[0014] Moreover, since the photography image was once changed into a middle image called a 
panorama image and the playback image was further compounded from the middle image, 
geometric conversion actuation had to be performed twice, the pixel information which the 
photography image had as a result was spoiled, and it had the technical problem that the image 
quality of a playback image deteriorated, 

[0015] This invention is displaying a high definition image on a high speed, while taking an 
example by the above-mentioned technical problem and enabling continuation migration of the 
view at the time of playback, and it aims at providing a user with the high image of presence. 
[001 6] 

[Means for Solving the Problem] The 1st means for attaining the purpose of this invention An 
image storage means to hold the image data of the image which photoed the perimeter by 
making into a view the point which draws the locus of arbitration and moves in the inside of 
space, A panorama image-sequences composition means to compound a panorama image for 
every view based on the image data which said image storage means holds, A view positional 
information setting means to set view positional information as the panorama image which said 
panorama image-sequences composition means compounded, It is characterized by providing a 
panorama image storage means to hold the panorama image data of the panorama image which 
said panorama image-sequences composition means compounded with the view positional 
information which said view positional information setting means set up. 

[0017] The 2nd means for attaining the purpose of this invention In the 1st means, the panorama 
image which said panorama image-sequences composition means compounded is divided into 
two or more images. A panorama image division means to set the space position relation 
centering on a view as each division image is provided. It is characterized by replacing the image 
data of the division image with which said panorama image division means created said panorama 
image storage means with a division panorama image storage means to hold with the view 
positional information which said view positional information setting means set up. 
[0018] The 3rd means for attaining the purpose of this invention An image storage means to hold 
the image data of the image which photoed the perimeter by making into a view the point which 
draws the locus of arbitration and moves in the inside of space. A view positional information 
setting means to set up view positional information to the image data which said image storage 
means holds. A photography image location means to set up the image positional information in 
the space centering on a view to the image data which said image storage means holds. It is 
characterized by providing a positional information storage means to hold the view positional 
information which said view positional information setting means set up. and the image positional 
information which said photography image location means set up. 

[0019] The 4th means for attaining the purpose of this invention Level and an image storage 
means to hold the image data of the image with which both vertical field angles made it move to 
arbitration, and photoed 1 80 degrees or more of a certain cameras in space, An image division 
means to divide the image for every photography time of day into two or more images to the 
image data which said image storage means holds, A view positional information setting means to 
set up view positional information for every photography time of day to the division image which 
said image division means makes, A division image location means to set up the image positional 
information in the space centering on a view to the division image which said image division 
means makes, It is characterized by providing a division photography image storage means to 
hold the division image which said image division means makes with the view positional 
information which said view positional information setting means sets up, and the image 
positional information which said photography image location means sets up. 
[0020] The 5th means for attaining the purpose of this invention A display image hysteresis 
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storage means to hold the view location of a display image, and the hysteresis of the direction of 
a look, A directions input means by which a user shows a direction to move and a direction to 
see to a display image. The view location of the display image which said display image 
hysteresis storage means holds, the view location of the image displayed on a degree from the 
amount of directions of the direction of a look, and said directions input means, and an image 
location count means to calculate the direction of a look. An image reading means to read the 
panorama image data in the view location for which said image location count means asked out 
of the image data which the 1st means holds. An image composition means to compound the 
display image of the direction of a look for which said image location count means asked from 
the panorama image data which said image reading means read, It is characterized by providing a 
display means to display the image which said image composition means compounded, and 
carrying out by repeating the input of the amount of directions by the user, composition of the 
image to it, and a display. 

[0021] The 6th means for attaining the purpose of this invention A display image hysteresis 
storage means to hold the view location of a display image, and the hysteresis of the direction of 
a look. The view location which the user will direct next based on the hysteresis of the view 
location which said positional information storage means holds, and the direction of a look, and 
an image location anticipation means to expect two or more sets of candidates of the direction 
of a look, An image reading means to read the panorama image data in two or more view location 
candidates whom said image location anticipation means expected out of the image data which 
the 1st means holds, A directions input means by which a user shows a direction to move and a 
direction to see to a display image, The view location of the display image which said display 
image hysteresis storage means holds, the view location of the image displayed on a degree from 
the direction of a look, and the contents of directions of said directions input means, and an 
image location count means to calculate the direction of a look. An image composition means to 
compound the display image of a view location and the direction of a look for which said image 
location count means asked from the panorama image data which said image reading means read, 
It is characterized by providing a display means to display the image which said image 
composition means compounded, and carrying out by repeating anticipation of the input of the 
amount of directions by the user, composition of the image to it, a display, and the image 
location of degree image. 

[0022] The 7th means for attaining the purpose of this invention In the 5th means, in the division 
image in the view location where said image location count means searched for said image 
reading means The image data of two or more images located near the direction of a look which 
said image location count means searched for It replaces with the image reading means read out 
of the image data which the 2nd means holds. It is characterized by replacing with an image 
composition means to compound the display image of the direction of a look for which said image 
location count means asked from two or more image data into which said image reading means 
read said image composition means. 

[0023] The 8th means for attaining the purpose of this invention In the 6th means, in the division 
image in the view location candidate said image location anticipation means expected said image 
reading means to be The image data of two or more images located near [ which said image 
location anticipation means expected ] the direction candidate of a look It replaces with the 
image reading means read out of the image data which the 2nd means holds. It is characterized 
by replacing with an image composition means to compound the display image of the view 
location where said image location count means searched for said image composition means 
from two or more image data which said image reading means read, and the direction of a look. 
[0024] The 9th means for attaining the purpose of this invention In the 5th means, in the 
photography image in the view location where said image location count means searched for said 
image reading means The image data of two or more images which took a photograph near the 
direction of a look which said image location count means searched for It replaces with the 
image reading means read out of the image data which the 3rd means holds. It is characterized 
by replacing with an image composition means to compound the display image of the direction of 
a look for which said image location count means asked from two or more image data into which 
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said image reading means read said image composition means. 

[0025] The 1 0th means for attaining the purpose of this invention In the 6th means, in the 
photography image in the view location candidate said image location anticipation means 
expected said image reading means to be The image data of two or more images which took a 
photograph near [ which said image location anticipation means expected ] the direction 
candidate of a look It replaces with the image reading means read out of the image data which 
the 3rd means holds. It is characterized by replacing with an image composition means to 
compound the display image of the view location where said image location count means 
searched for said image composition means from two or more image data which said image 
reading means read, and the direction of a look. 

[0026] The 1 1th means for attaining the purpose of this invention In the 5th means, in the 
division image in the view location where said image location count means searched for said 
image reading means The image data of two or more division images with which it is reflected 
near the direction of a look which said image location count means searched for It replaces with 
the image reading means read out of the image data which the 4th means holds. It is 
characterized by replacing with an image composition means to compound the display image of 
the direction of a look for which said image location count means asked from two or more image 
data into which said image reading means read said image composition means. 
[0027] The 1 2th means for attaining the purpose of this invention In the 6th means, in the 
division image in the view location candidate said image location anticipation means expected 
said image reading means to be The image data of two or more division images with which it is 
reflected near [ which said image location anticipation means expected ] the direction candidate 
of a look It replaces with the image reading means read out of the image data which the 4th 
means holds. It is characterized by replacing with an image composition means to compound the 
display image of the view location where said image location count means searched for said 
image composition means from two or more image data which said image reading means read, 
and the direction of a look. 

[0028] The 1 3th means for attaining the purpose of this invention In the view location and the 
direction of a look which said image location count means searched for [ said image composition 
means ] in the 5-1 2th means When the direction of a look is specified that a view location 
moves perpendicularly to the direction of a look in accordance with the direction of a look of the 
display image in this time it is on the same flat surface as the display image in this time, and is 
the same as the movement magnitude of a view in an image core — it is characterized by 
compounding the image in the location made to *3i«3Wc**. 

[0029] The 14th means for attaining the purpose of this invention A panorama image storage 
means to hold the time series panorama image which carried out fixed time amount photography 
of the image of the perimeter in one view, A display image hysteresis storage means to hold the 
hysteresis of the direction of a look of a display image, and a directions input means by which a 
user shows a direction to see to a display image. An image location count means to calculate the 
direction of a look of the display image which said display image hysteresis storage means holds, 
and the direction of a look of the image displayed on a degree from the contents of directions of 
said directions input means. An image reading means to read the image data of the panorama 
image in the next time of day of the image currjently displayed from said panorama image storage, 
An image composition means to compound the display image of the direction of a look for which 
said image location count means asked from the panorama image data which said image reading 
means read, It is characterized by providing a display means to display the image which said 
image composition means compounded, and carrying out by repeating composition of the 
directions input by the user, and the image corresponding to it, and a display. 
[0030] The 1 5th means for attaining the purpose of this invention is characterized by arranging 
the image data of the image with which a view location and the direction of a look continue on 
the recording track with which a record medium continues or adjoins to the perimeter image 
divided in the direction of plurality for every view. 

[0031] The 16th means for attaining the purpose of this invention is characterized by recording 
the image data of the image sequences for every direction on a separate record medium. 
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interlocking the head to all record media to the image data of an image which divided the 
perimeter image in two or more directions, at the time of read-out of image data, and making it 
move. 

[0032] The 17th means for attaining the purpose of this invention is characterized by recording 
the image data of the image of the same view on the same location on a record medium in the 
16th means. 

[0033] In the 1 5th, 1 6, and 1 7 means, the 1 8th means for attaining the purpose of this invention 
compresses image data, and is characterized by arranging the size of all image data with the 
maximum data size in it. 
[0034] 

[Function] As opposed to the image data of the image which photoed the perimeter by making 
into a view the point which according to this invention with such a description draws the locus of 
arbitration and moves in the inside of the space which an image storage means holds While the 
panorama image-sequences composition means compounds the panorama image centering on a 
view beforehand By an image composition means' compounding the location image according to 
it from a panorama image, and repeating what a display means displays to the migration direction 
of a view and the direction of a look where a user uses and directs a directions input means at 
the time of playback The panorama image equipment which provides a user with the high image 
of presence is realizable. 

[0035] Moreover, using the hysteresis information on a view location and the direction of a look 
which a display image hysteresis storage means holds, an image location anticipation means 
expects the migration direction which the user will input next, and can realize the image 
equipment which performs high-speed image reconstruction by reading the image data which 
needs an image reading means beforehand based on the result 

[0036] Moreover, an image composition means can realize the panorama image equipment which 
offers a high definition image by compounding a direct presentation image from a photography 
image. 

[0037] Moreover, since an image division means can make small time amount which divides a 
panorama image and requires an image reading means for read-out of image data by reading only 
a partial image required for image composition, the panorama image equipment which performs 
high-speed image reconstruction Is realizable. 

[0038] Moreover, the image photoed with the camera with a big field angle is divided, and the 
panorama image equipment which reproduces a high definition image at a high speed can be 
realized by performing image composition using the image. 

[0039] Moreover, since an image store can make small time amount which read-out of image 
data takes by controlling two or more read-out heads while arranging image data the optimal on 
an are recording medium, the panorama image equipment which performs high-speed image 
reconstruction is realizable. 
[0040] 

[Example] Hereafter, the example of this invention is explained, referring to a drawing. 

[0041] (Example 1) Drawing 1 shows the configuration of the first example realized using the first 

means and second means of this invention. 

[0042] In drawing 1 , perimeter photography equipment 101 divides and photos the sight of all 
the directions seen from a certain view in two or more images, and it photos it continuously, 
moving using a truck. 

[0043] An A/D-conversion means digital-data-izes the image which perimeter photography 
equipment 101 photoed 

[0044] The image amendment means 103 performs amendment of distortion with the lens of a 
camera, and amendment of brightness to the digitized image data. The image storage means 104 
memorizes the amended image data. 

[0045] The panorama image-sequences composition means 1 05 compounds the panorama image 
in the view using the division image of the perimeter which perimeter photography equipment 101 
photoed in the same view out of the image data which the image storage means 104 holds. 
[0046] To the panorama image in each view which the panorama image-sequences composition 
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means 105 compounded, the view positional information setting means 106 determines the 
position coordinate of each view in space, and is set up. 

[0047] The panorama image storage means 1 07 will return the panorama image of the view 
corresponding to it, if the image data of the panorama image which the panorama image 
composition means 105 compounded, and the view location number which the view positional 
information setting means 106 assigned are accumulated and a view location number is given. 
[0048] The directions input means 108 inputs a view and the relative migration direction of a 
look, while a user looks at a display image at the time of playback of an image. 
[0049] The view location of the image which the display image hysteresis storage means 1 09 is 
based on directions of a user, and is compounded and displayed serially, and the hysteresis 
information on the direction of a look are held. 

[0050] The image location count means 110 calculates the view location of the image which 
should be displayed on the next moment, and the value of the direction of a look from the 
movement magnitude of the relative view which the user inputted as the view of an image on 
display, and the value of the direction of a look using the directions input means 108 by this time 
which the display image hysteresis storage means 1 09 holds, and the direction of a look, 
[0051] The image reading means 1 1 1 reads the image data of the panorama image in the view 
location which the image location count means 1 1 0 calculated from the panorama image storage 
means 107. 

[0052] The image composition means 1 1 2 compounds the image of the direction of a look which 
the image location count means 1 1 0 calculated based on the panorama image data which the 
image reading means 111 read. 

[0053] The display means 113 displays the image which the image composition means 112 
compounded. Here, the relation between a photography image and a panorama image is 
described. 

[0054] Perimeter photography equipment 1 01 has a configuration as shown for example, in 
drawing 2 -a. As for a reflecting mirror and 202, in drawing 2 -a, 201 is [ the camera section and 
203 ] trucks. 

[0055] A reflecting mirror 201 has the configuration where the plane mirror was stuck on the 
outside of the side face with the base of n square shape of a forward multiple drill. 
[0056] The camera section 202 consists of n cameras, and each camera photos the image of a 
different direction from an own optical axis with the plane mirror with which a reflecting mirror 
201 corresponds, respectively. 

[0057] The inclination from the shaft of the side face of a reflecting mirror 201 is made into 45 
degrees, an optical axis arranges each camera of the camera section 202 so that it may become 
equal to the distance to which the distance of the shaft, parallel and optical axis, and shaft of a 
reflecting mirror 201 met the optical axis of a lens core and a corresponding reflecting mirror, 
and the image which looked at the perimeter is divided and photoed with n cameras from one 
view on the shaft of a reflecting mirror 201 . 

[0058] Drawing 2 -b shows the physical relationship of the two or more division images and the 
view which were photoed with perimeter photography equipment 1 01 . 

[0059] 204 is a view and 205 is the photography image of each direction. The photography image 
205 forms the side face of the multiple column of n square shape centering on a view 204. 
[0060] The synchronization of all the cameras of the camera section 202 is taken, and the image 
of the perimeter of the same time of day is photoed. Furthermore, a photograph is taken moving 
perimeter photography equipment 101 using a truck 203, and the animation of all the directions 
which a view moves continuously is photoed. 

[0061] Drawing 3 is drawing having shown the physical relationship of a photography image and a 
panorama image. As for a photography image and 302. in drawing 3 , 301 is [ a panorama image 
and 303 ] the projection fields of a photography image. 

[0062] A panorama image is set up as the spherical surface with a certain radius r in the 
rectangular coordinate system centering on a view. The location of the pixel P on a panorama 
image makes Z shaft orientations a zero, and expresses them by the X~Z flat surface, the Y-Z 
flat surface, and the spherical coordinate system using angle thetap made, respectively and phip. 
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Moreover, the straight line which connects an image core to a view expresses the location based 
on C of each photography image / I ] images by the spherical coordinate system using the 
distance f of the angle thetai, phii, and the view and the photography image made with a X-Z flat 
surface and a Y-Z flat surface, respectively, i.e., the focal distance at the time of photography. 
At this time, (thetai. phii) become the direction of a look of a photography image. 
[0063] Since the physical relationship between each photography image a view and around the 
view is eternal in all the views that took a photograph, it finds the direction of a look of each 
photography image, and the focal distance by performing a calibration beforehand. At this time, 
the travelling direction of the perimeter photography equipment 101 at the time of photography 
is made in agreement with the direction of a zero of a panorama image. Then, the perspective 
projection centering on a view is performed from the photography image 301 to up to the 
panorama image 302 by making into a parameter the direction of a look and focal distance which 
were found. Thereby, the value of the pixel of the projection field 303 on the panorama image 
302 is determined. This reverse conversion is performed at the time of image reconstruction. 
That is, the perspective projection of the pixel on a panorama image is carried out on the display 
image side set up all over space. 

[0064] Thus, the image data of a panorama image is compounded by projecting the photography 
image of all the directions to up to the panorama image 302 in one view. Panorama image 
sequences are completed by carrying out to all the views that photoed this composition. 
[0065] Since photography is performed moving perimeter photography equipment 1 01 , the view 
of a photography image is also moved continuously and one locus is drawn in space. 
[0066] The view positional information setting means 106 shakes a number at the panorama 
image which the panorama image composition means 105 compounded in order according to the 
list of the view on this locus. This sen/es as information which manages a view location. For 
example, the number of the panorama image of a top view is set to 1 , and the number increased 
every [ 1 ] in order is assigned to the panorama image following a degree. 

[0067] Next, the processing at the time of image reconstruction is described in detail. Drawing 4 
shows composition of the display image at the time of image reconstruction, and the flow of 
display processing. Hereafter, the detail is explained in accordance with the flow of the step of 
drawing 4 . 

[0068] (step S 1-1) It is the processing loop formation which repeats composition of the image 
according to a user s input to a display image, and a display, and performs them. 
[0069] (step S 1-2) At the initiation time of a processing loop formation, the view of the head of 
a path and a travelling direction are specified as the view of a display image, and a direction of a 
look, respectively. After it, the view location and the direction of a look of the image displayed on 
a degree are determined [ the view location of the image which the image location count means 
110 shows by this time which the display image hysteresis storage means 109 holds, the 
information on the direction of a look, and / that the user inputted using the directions input 
means 108 / relative ] from migration. If there is especially no assignment by the directions input 
means 108, the view location of the display image in this time and the direction of a look will be 
specified as it is. The image location count means 110 accumulates a count result in the display 
image hysteresis storage means 109 again. 

[0070] (step S 1-3) Step S The image reading means 111 reads the image data of the panorama 
image corresponding to the view location which the image location count means 1 10 determined 
by 1 -2 from the panorama image storage means 1 07. 

[0071] (step S 1-4) The image composition means 112 compounds the image of the direction of 
a look which the image location count means 110 determined from the image data of the 
panorama image which the image reading means 111 read: 

[0072] (step S 1-5) The display means 113 displays the image which the image composition 
means 112 compounded. 

[0073] (step S 1-6) The direction which a user wants to see the image which the display means 
1 1 3 displays and to see next using the directions input means 1 08 is specified. 
[0074] (step S 1-7) Processing is again returned to step S1-1, and processings from step SI -2 
to step 81-6 are repeated. Step S If a user directs termination of processing by 1-6, it will 
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escape from a processing loop formation and the whole processing will be ended. 
[0075] Step S In 1-6. the item which a user can specify with the directions input means 108 is 
advance, retreat, a halt, a RRC, a RLC, upper rotation, bottom rotation, expansion, contraction, 
or termination. In these, three, advance, retreat, and a halt, are processed as what the same item 
is continuing being specified as, unless a user will specify another item in the three after next 
time, if three ones of the items is specified once. A change of view migration and the direction of 
a look can be made to coincidence by specifying rotation actuation further in this condition. On 
the other hand, about expansion and contraction, only when it is a view location and the idle 
state to which the direction of a look was fixed, assignment is made possible. In addition, about 
rotation of the vertical direction, expansion, and contraction, based on the photographic 
coverage of a photography image, and resolution, the upper limit is set up beforehand, and only 
when falling within the range of it, it considers as the thing which a user can specify. 
[0076] Detail of the contents of processing of the image location count means 109 in step SI -2 
over each of these actuation items is given. Here. 1 - m shall be shaken for the view of a 
panorama image at those with m point, and the panorama image for every view as a view location 
number. Moreover, the view location number of the display image in this time which the display 
image hysteresis storage means holds is set to n, and (theta. phi), and a focal distance are set to 
f for the direction of a look. Moreover, the direction of a look and unit variation of a focal 
distance are set to delta theta. delta phi. and delta f. respectively. 

[0077] (a) Advance : advance one view location. That is, the view location number of the 
following display image is set to n+1. However, to the direction of a zero of a panorama image, 
i.e., the travelling direction at the time of photography, if the direction of a look of the display 
image in this time is a larger value than 90 degrees, it will return one view location and will set 
the view location number of the following display image to n-1 . This means showing the image 
which goes to hard flow to the travelling direction at the time of photography. About the 
direction of a look, and a focal distance, it considers as as [ the image which is indicating by 
current ]. 

[0078] (b) Retreat : return one view location and set the view location number of the following 
display image to n-1. However, to the direction of a zero of a panorama image, i.e.. the travelling 
direction at the time of photography, if the direction of a look of the display image in this time is 
a larger value than 90 degrees, it will advance one view location and will set the view location 
number of the following display image to n+1 . About the direction of a look, and a focal distance, 
it considers as as [ the image which is indicating by current ]. 

[0079] (c) Halt : consider a view location, the direction of a look, and a focal distance as as [ the 
value of the display image in this time ]. 

[0080] (d) RRC : consider a view location and a focal distance as as [ the display image in this 
time ]. However, in being in the condition that advance or retreat is already specified, according 
to processing of (a) and (b), it changes a view location. The direction of a look is changed into 
having shifted only unit include-angle deltatheta to the direction of a look in this time (theta- 
delta theta and phi). 

[0081] (e) RLC : consider a view location and a focal distance as as [ the display image in this 
time ]. However, in being in the condition that advance or retreat is already specified, according 
to processing of (a) and (b), it changes a view location. The direction of a look is changed into 
having shifted only unit include-angle deltatheta to the direction of a look in this time 
(theta+delta theta and phi). 

[0082] (f) Upper rotation : consider a view location and a focal distance as as [ the display image 
in this time ]. However, in being in the condition that advance or retreat is already specified, 
according to processing of (a) and (b), it changes a view location. The direction of a look is 
changed into having shifted only unit include-angle deltatheta to the direction of a look in this 
time (theta, phi+deltaphi). 

[0083] (g) Bottom rotation : consider a view location and a focal distance as as [ the display 
image in this time ]. However, in being in the condition that advance or retreat is already 
specified, according to processing of (a) and (b), it changes a view location. The direction of a 
look is changed into having shifted only unit include-angle deltatheta to the direction of a look in 
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this time (theta. phi-deltaphi). 

[0084] (h) Expansion : consider a view location and the direction of a look as as [ the display 
image in this time ]. A focal distance is changed into f*-delta f. 

[0085] (i) — contraction: — a view location and the direction of a look are considered as as 
[ the display image in this time ]. A focal distance is changed into f-delta f. 

[0086] Next to the view location which carried out in this way and was determined, the direction 
of a look, and a focal distance, the image composition means 112 considers the space centering 
on the view location, and sets up the direction of a look searched for in it, and the imagination 
image side which has an image core in the location of a focal distance. To this virtual image side, 
a perspective projection is performed from the panorama image in this view, and the value of the 
pixel of a virtual image side is determined. In this way, let the compound image be the image 
displayed on a degree. 

[0087] Since the imagination migration in space can be experienced by compounding the 
panorama image in the view which continued beforehand using the image equipment which 
carried out the above configurations according to this example as mentioned above, and 
repeating compounding and displaying the image to which the view location and the direction of a 
look were changed continuously according to directions of a user at the time of playback, the 
presence of an image raises. 

[0088] In addition, although the spherical surface centering on a view expressed the panorama 
image in this example, the same effectiveness is acquired even if it uses the cylinder side 
centering on a view. 

[0089] (Example 2) Drawing 5 is the block diagram of the second example realized using the first 
means and second means of this invention. 

[0090] In drawing 5 , perimeter photography equipment 501 is divided and photoed in two or 
more images which are made to rotate a camera for the sight of all the directions seen from a 
certain view, and have a lap mutually, and photos the perimeter image in the view which 
continued while moving using the truck. 

[0091] The A/D-conversion means 102 digital-data-izes the image which perimeter photography 
equipment 501 photoed. 

[0092] The image amendment means 103 performs amendment of distortion with the lens of a 
camera, and amendment of brightness to the digitized image data. 

[0093] The image storage means 104 memorizes the amended image data: The panorama image- 
sequences composition means 505 compounds the panorama image in the view using the division 
image of the perimeter which perimeter photography equipment 501 photoed in the same view 
out of the image data which the image storage means 104 holds. 

[0094] To the panorama image in each view which the panorama image-sequences composition 
means 505 compounded, the view positional information setting means 106 determines the 
position coordinate of each view in space, and is set up. 

[0095] The panorama image storage means 1 07 memorizes the image data of the panorama 
image in all the photoed view locations with view positional information. 
[0096] Perimeter photography equipment 501 has a configuration as shown for example, in 
drawing 6 -a. As for a camera and 602, in drawing 6 -a, 601 is [ a rotation plinth and 603 ] 
trucks. A camera 601 is attached in the rotation plinth 602. The rotation plinth 602 performs 
rotation of a pan and a tilt a core [ the camera core of a camera 601 ], i.e., a core [ location / to 
a photography image / view ]. This photos the image which divided the perimeter into plurality 
centering on the view. At this time, it is made to rotate so that some ****** images may 
overlap, and a photograph is taken. 

[0097] Drawing 6 -b indicates the physical relationship of a view to be the image photoed with 
perimeter photography equipment 501 . In drawing 6 , 604 is a view and 605 is the photography 
image of each direction. When a photograph is divided and taken in the direction of n, the 
photography image 605 forms the side face of the multiple column of n square shape centering 
on a view 604. 

[0098] After rotating a camera and photoing the perimeter of one view, only minute distance 
moves using a truck 603 and the image of all the directions of the view which repeats and 
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continues photoing a perimeter there again is photoed. 

[0099] The panorama image-sequences composition means 505 compounds the panorama image 
in each view which perimeter photography equipment 501 photoed using the image data which 
the image storage means 1 04 photoed as mentioned above holds. 

[0100] A photography image once sets up a suitable direction and performs a perspective 
projection on a panorama image. In the projection image of an adjoining photography image, it 
laps shifting a relative position, correlation of the pixel value of a field is searched for, and the 
relative position where correlation becomes high most is determined. A panorama image is 
generable by determining such a relative position to the projection image of all the images that 
photoed the perimeter of one view. 

[0101] The panorama image-sequences composition means 505 extracts a motion of the focus 
in an image between the panorama images of the view which adjoined further, and detects the 
focus group which moves toward a radiation-like outside focusing on a certain point The 
projection field on a panorama image is shifted so that the direction of a straight line to which 
the point which has taken the lead in migration of this focus was connected may be made into 
the travelling direction of the photography equipment at the time of photography and this 
direction may lap with the Z-axis from a view. 

[0102] Thus, the panorama image sequences of a continuation view are generated by 
compounding the panorama image for every view, and determining a travelling direction between 
the panorama images of a continuous view. 

[0103] Since the panorama image sequences of the view which continued by performing 
processing which was described above to the image which was made to rotate a camera and was 
photoed are generable as mentioned above according to this example, it becomes possible to 
compound and display the image to which the view and the direction of a look were changed 
continuously based on the image data at the time of playback, and it can raise the presence of 
an image. 

[01 04] In addition, in this example, in order to know the location of the photography image to a 
view, correlation between images was used, but the actuation information on the camera at the 
time of photography is recorded by the sensor etc., and the same effectiveness is acquired even 
if it uses it directly. 

[0105] (Example 3) Drawing 7 is the block diagram of the third example realized using the second 
means of this invention, the seventh means, and the eighth means. 

[0106] In drawing 7 , the image storage means 104 memorizes the image data which photoed all 
the directions around the view moved continuously. 

[0107] The panorama image-sequences composition means 105 compounds the panorama image 
in the view using the image data of the photography image in the same view out of the image 
data which the image storage means 104 holds. 

[0108] To the panorama image in each view which the panorama image-sequences composition 
means 105 compounded, the view positional information setting means 106 determines the 
position coordinate of each view in space, and is set up. 

[0109] The panorama image division means 701 divides into two or more images the panorama 
image for every view which the panorama image-sequences composition means 105 
compounded. 

[0110] The division panorama image storage means 702 will choose and return the division image 
data in which the photography data of the direction are contained, if the image data of the 
panorama image which the panorama image division means divided is memorized with the view 
positional information which the view positional information setting means set up and a view 
location and the direction of a look are specified. 

[0111] The view location of the image which the display image hysteresis storage means 109 is 
based on directions of a user, and is compounded and displayed serially, and the hysteresis 
information on the direction of a look are held. 

[0112] The image location anticipation means 703 predicts the view location which the user will 
direct next, and the direction of a look based on the hysteresis information on the display image 
which the display image hysteresis storage means 109 holds. 
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[01 13] The image reading means 704 reads the view location which the image location 
anticipation means 703 expected, and the image with which the image of the direction of a look 
is reflected from the division panorama image storage means 702. 

[0114] The directions input means 108 inputs a view and the relative migration direction of a 
look, while a user looks at a display image at the time of playback of an image. 
[01 1 5] The image location count means 1 1 0 calculates the view location of the image which 
should be displayed on the next moment, and the value of the direction of a look from the 
movement magnitude of the relative view which the user inputted as the view of an image on 
display, and the value of the direction of a look using the directions input means 108 by this time 
which the display image hysteresis storage means 109 holds, and the direction of a look. 
[0116] The image composition means 705 compounds the image of the direction of a look which 
the image location count means 110 calculated based on the division panorama image data which 
the image reading means 704 read. 

[01 17] The display means 113 displays the image which the image composition means 705 
compounded. Here, the processing which divides a panorama image is described. 
[0118] Drawing 8 is the conceptual diagram of the image division which the panorama image 
division means 701 performs. In drawing 8 . 801 is a panorama image and 802 is a division image. 
[0119] To the panorama image of the shape of the spherical surface which the panorama image- 
sequences composition means 105 compounded, the division-into-equal-parts rate of the 
perimeter of the direction of theta is carried out to k pieces, and the division image of k sheets 
is generated. Division is started on the basis of the Z-axis. and size of pix(2xi-1)/k [rad] and the 
direction of theta is made into 2x pi/k [rad] for the direction of an image core in the division 
image of No. i. The coordinate based on [ this ] images is used as a direction of a look of this 
division image. Thus, the image data of the divided image is accumulated in the division panorama 
image storage means 702 with view positional information and the direction data of a look. 
[01 20] Next, the processing at the time of image reconstruction is described in detail. Drawing 9 
shows the flow of the processing at the time of the image reconstruction in this example. 
Hereafter, the detail is explained along with the step of drawing 9 . 

[0121] (step S 3-1) The image reading means 704 reads the image data to which the direction is 
reflected from the division panorama image storage means 702 by making a travelling direction 
into the direction of a look, making the initiation point of a path as a view location, the image 
composition means 705 compounds a display image from the image data, and the display means 
113 displays. 

[0122] (step S 3-2) It is the processing loop formation which repeats composition of the image 
according to a user's input, and a display, and performs them. 

[01 23] (step S 3-3) The direction which a user wants to see the image which the display means 
113 displays and to see next using the directions input means 1 08 is specified. 
[0124] (step S 3-4) The image location count means 110 determines [the view location of the 
display image in this time which the display image hysteresis storage means 1 09 holds, the 
information on the direction of a look, and / that the user inputted using the directions input 
means 108 / relative ] from migration the view location and the direction of a look of the image 
displayed on a degree. If there are especially no directions by the directions input means 108. 
the view location of the display image in this time and the direction of a look will be specified as 
it is. The image location count means 110 accumulates a count result in the display image 
hysteresis storage means 109 again. 

[01 25] (step S 3-5) The image location anticipation means 703 predicts the view location of the 
image for which the user will ask next, and the direction of a look based on the view location of 
the display image to current which the display image hysteresis storage means holds, and the 
hysteresis of the direction of a look. 

[01 26] (step S 3-6) The image with which the direction where the image reading means 704 was 
specified based on the potential result of the image location anticipation means 703 is reflected 
is read from the division panorama image storage means 702. 

[01 27] (step S 3-7) The image composition means 705 compounds the view location specified by 
the image location count means 110, and an image with the direction of a look from the image 
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data which the image reading means 704 read 

[01 28] (step S 3-8) The image which the image composition means 705 compounded is 
displayed. 

[01 29] (step S 3-9) Processing is again returned to step S3-2, and processings from step S3-3 
to step S3-8 are repeated Step S If a user directs termination of processing by 3-3, it will 
escape from a processing loop formation and the whole processing will be ended. 
[01 30] Step S The processing in 3-5, and 6 and 7 is described in more detail. Drawing 10 table- 
izes the regulation for expecting the view location of the image which the image location 
anticipation means 703 displays on a degree, the direction of a look, and a focal distance. Here, 
the image which displays the image which displayed the image on display at the time before I (t) 
and one at present at the time of I (t-1) and a degree is set to I (t+l). Moreover, the view 
location number of I (t) is set to n. 

[01 31] The case where the view location number in I (t-1) is n-1 as one example of the 
regulation shown in drawing 10 is explained. 

[01 32] First if I (t-1) and I (t) compare a view location number, the number of 1 is increasing. 
This shows that the view migration by the last user actuation is "advance." In this case, it is 
either while changing the direction of a look, continuing "advance" which continues "a halt" of 
view migration, and "advance" as actuation which a user can perform (nothing is operated). 
[01 33] Therefore, the actuation expected, the view location at that time, and the combination of 
the direction of a look are (1 ) halt. : View location: n, the direction of a look : (thetat, phit) 

(2) Advance: View location: n+1 , the direction of a look : (thetat. phit) 

(3) Advance + PLC: View location: n+1. the direction of a look : (thetat+deltatheta, phit) 

(4) Advance + RRC: View location: n+1, the direction of a look : (thetat-deltatheta. phit) 

(5) Rotation on advance +: View location: n+1, the direction of a look : (thetat. ph'rt+deltaphi) 

(6) Rotation under advance +: View location: n+1, the direction of a look : (thetat, phit-deltaphi) 
They are six kinds of 

[01 34] The image reading means 704 reads all the image data of the division panorama image 
with which the image in the view location and the direction of a look which were expected in this 
way is reflected. When the image data read in case the display image in this time is already 
compounded can use again, it leaves the data as it is. and does not carry out newly reading. 
[01 35] To the view location and the direction of a look which the image location count means 
109 determined, the image composition means 705 considers the space centering on the view 
location, and sets up an imagination image side in the direction of a look searched for further. In 
the image data which the image reading means 704 read, the multiple selection of the view 
location which the image location count means 109 determined, and the image data to which the 
direction of a look is reflected is made, and those image data is projected by the perspective 
projection on a virtual image side. In this way, the following display image is compounded. 
[01 36] Since only the image data of a field required at the time of playback can be alternatively 
read by dividing and memorizing a panorama image as mentioned above according to this 
example, image reconstruction processing can be performed at a high speed 
[01 37] Moreover, since image data required for image composition can be beforehand read by 
predicting the actuation which the user will perform to the degree based on the view location of 
the display image by a user's actuation result, and the hysteresis of the direction of a look, image 
reconstruction processing can be performed at a high speed. 

[01 38] In addition, although the panorama image was divided only about the direction of theta in 
this example, if it divides also about the direction of phi, since the image data per sheet more 
can be made small and selectivity will increase, image reconstruction processing can be 
performed at a high speed. 

[01 39] Moreover, in this example, although the spherical-surface-like panorama image was 
divided, the same effectiveness is acquired by dividing into the image of the direction of plurality 
similarly to a cylinder-like panorama image. 

[0140] Moreover, when a display image was compounded using a division panorama image, 
directions by the user were expected, but in this example, also when compounding a display 
image using a panorama image, a photography image, and the image that divided the photography 
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image further, directions by the user are expected beforehand, it is reading required data 
previously and the same effectiveness is acquired. This is realized by the sixth, tenth, and 
twelfth means of this invention. 

[0141] (Example 4) Drawing 1 1 shows the configuration of the fourth example realized using the 
third means and ninth means of this invention. 

[01 42] In drawing 1 1 , the image storage means 1 04 memorizes the image data which photoed all 
the directions around the view moved continuously. 

[0143] To each photography image for every view which the image storage means 104 holds, the 
view positional information setting means 106 determines the position coordinate of each view in 
space, and is set up. 

[0144] To each photography image for every view which the image storage means 104 holds, the 
photography image location means 1 101 determines the image location in the space centering on 
a view location, and is set up. 

[0145] The positional information storage means 1 102 accumulates the view positional 
information which the view positional information setting means 106 set up, and the image 
positional information which the photography image location means 1101 set up for every 
photography image which the image storage means 104 holds. 

[0146] The view location of the image which the display image hysteresis storage means 109 is 
based on directions of a user, and is compounded and displayed serially, and the hysteresis 
information on the direction of a look are held. 

[0147] The directions input means 108 inputs a view and the relative migration direction of a 
look, while a user looks at a display image at the time of playback of an image. 
[0148] The image location count means 110 calculates the view location of the image which 
should be displayed on the next moment, and the value of the direction of a look from the 
movement magnitude of the relative view which the user inputted as the view of an image on 
display, and the value of the direction of a look using the directions input means 108 by this time 
which the display image hysteresis storage means 109 holds, and the direction of a look. 
[0149] To the view location and the direction of a look which the image location count means 
110 determined, the image reading means 1103 determines which photography image should be 
read from the view positional information and image positional information of the photography 
image which the positional information storage means 1102 holds, and reads the image data of a 
required image from the image storage means 104. 

[01 50] The image composition means 1 104 compounds the image of the view location which the 
image location count means 110 determined, and the direction of a look from the image data 
which the image reading means 1 103 read. 

[0151] The display means 113 displays the image which the image composition means 1 104 
compounded. Here, since it asks when taking a photograph using perimeter photography 
equipment as shown in drawing 2 , and the migration direction of photography equipment is used 
as a reference axis and the bearing of the exposure axis (theta, phi) of each camera centering on 
a view and a focal distance f perform a calibration beforehand, these values are used as 
positional information of the photography image in the space centering on a view. 
[0152] The shadow image location means 1101 sets up this image positional information to the 
photography image in all photography points. Moreover, the field angle of a photography image is 
also set as coincidence as one of the image positional information. 

[0153] Thus, it is immediately called for by giving the information on an image location and a field 
angle for every photography image which field in the space centering on a view each image has 
the information on. 

[0154] On the other hand, the image reading means 1103 calculates whether the information on 
which field in the space centering on a view is required using the field angle of a display image to 
the view location of the display image which the image location count means 1 10 determines, 
and the direction of a look. Furthermore, it determines as an image which uses two or more 
photography images with which the field for which it asked, and a photography field lap based on 
the positional information of the photography image which the positional information storage 
means 1 1 02 holds in order to compound a display image. 
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[01 55] Drawing 12 shows the physical relationship of the photography image in the image 

composition which the image composition means 1 104 performs, and a virtual image. 

[0156] In drawing 12 . 1201 is the field of the photography image with which a photography image 

and 1202 are projected on a virtual image, and 1203 is projected on a virtual image. 

[01 57] The image composition means 1 1 04 is the direction of a look which the image location 

count means 110 determines, and sets the virtual screen 1202 as the location from which only 

the focal distance beforehand set up as a parameter of a display image was separated. To the 

field 1203 on the photography image 1201. the perspective projection centering on a view is 

performed and the value of each pixel on the virtual screen 1202 is determined In this way, let 

the compound virtual screen 1202 be the following display image. 

[0158] Since the count of image transformation can be reduced compared with the case where a 
panorama image is used by holding the positional information of a photography image and 
compounding a direct presentation image from a photography image based on the positional 
information, as mentioned above according to this example, the image quality of the image to 
display can be raised, 

[0159] In addition, although this example explained the case of the image photoed using the 
photography equipment using a plane mirror, the same effectiveness is acquired even if it uses 
the photography image which was made to rotate a camera and was photoed. In this case, at the 
time of photography, the angle of rotation is measured and it determines the location of a 
photography image. Or a photography image may once be projected on a panorama image side, 
mutual physical relationship may be determined, and the positional information may be used. 
[0160] (Example 5) Drawing 13 shows the configuration of the fifth example realized using the 
thirteenth means of this invention. 

[0161] In drawing 13 , the panorama image storage means 107 memorizes the panorama image- 
sequences data in the view moved continuously. 

[01 62] The directions input means 1 08 inputs a view and the relative migration direction of a 
look, while a user looks at a display image at the time of playback of an image. 
[0163] The view location of the image which the display image hysteresis storage means 109 is 
based on directions of a user, and is compounded and displayed serially, and the hysteresis 
information on the direction of a look are held. 

[0164] The image location count means 1 10 calculates the view location of the image which . 
should be displayed on the next moment, and the value of the direction of a look from the 
movement magnitude of the relative view which the user inputted as the view of an image on 
display, and the value of the direction of a look using the directions input means 108 by this time 
which the display image hysteresis storage means 109 holds, and the direction of a look. 
[01 65] The image reading means 1 1 1 reads the image data of the panorama image in the view 
location which the image location count means 110 determined from the panorama image storage 
means 107. 

[0166] The image composition means 1301 compounds a display image from the image data 
which the image reading means 111 read. 

[01 67] The display means 1 1 3 displays the image which the image composition means 1 301 
compounded. The image composition processing which the image composition means 1301 
performs is explained in detail. 

[0168] Drawing 14 shows the physical relationship of the image at the time of the image 
composition which the image composition means 1301 performs. 

[01 69] In drawing 14 . the view of an image that a panorama image and 1402 should be displayed 
on the view of a panorama image and the display image in this time, and 1401 should display 
1403 on a degree, the display image in this time [ 1404 ], and 1405 are the images which should 
be displayed on a degree. 

[0170] The directions input means 108 can direct migration right and left to the direction of the 
present look besides rotation of advance, retreat, and the direction of four directions. As for n' 
and the direction of a look, a view location is set to thetan by the count result of the image 
location of the following display image by the image location count means 1 1 0. when In and a 
view location number are set to n. the direction of a look was set to thetan and migration 
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leftward is directed in the display image in this time. Here, the distance of View n and n' is the 
unit distance set up beforehand, and sets this with I. 

[01 71] Since view n' is in the location from which it separated from the locus of the view at the 
time of photography, there is no panorama image in this view location in the panorama image 
storage means 1 07. Therefore, the image composition means 1 301 compounds the image in view 
n' in approximation using the panorama image in View n. 

[0172] On the flat surface containing Image In, image In' which has an image core in the location 
which separated only I from the image core of Image In in the direction of the left is set up. The 
perspective projection of the field where the panorama image in View n corresponds is carried 
out from View n to this image In'. In this way, let compound image in* be an approximation image 
in view n'. 

[01 73] Since an image can be changed according to this example as mentioned above as if it 
moved to the view by compounding an image in approximation from the image data of existing 
another view even when a view without panorama image data is specified, the presence of an 
image can be raised. 

[01 74] (Example 6) Drawing 15 shows the configuration of the sixth example realized using the 
fourth means and eleventh means of this invention. 

[0175] In drawing 15 , both the perimeter photography equipments 1501 are continuously 
photoed, while the field angle of level and a perpendicular direction photos the image of 1 80 
degrees or more and moves using a truck. 

[0176] An A/D-conversion means digital-data-izes the image which perimeter photography 
equipment 1501 photoed, and the image storage means 104 accumulates the image data. 
[01 77] The image division means 1 502 divides into two or more images the image which the 
image storage means 104 holds. 

[0178] The view positional information setting means 106 sets up the view location of each 
image in space to the image which the image division means 1 502 divided. 
[01 79] The division image location means 1 503 sets up the location of the image in the space 
centering on a view to the image which the image division means 1 502 divided, respectively. 
[0180] The division image storage means 1504 is accumulated with the view positional 
information to which the view positional information setting means 1 06 set the image data of the 
image which the image division means 1 502 divided, and the image positional information which 
the division image location means 1 503 set up. 

[01 81] The directions input means 108 inputs a view and the relative migration direction of a 
look, while a user looks at a display image at the time of playback of an image. 
[0182] The view location of the image which the display image hysteresis storage means 109 is 
based on directions of a user, and is compounded and displayed serially, and the hysteresis 
information on the direction of a look are held. 

[0183] The image location count means 110 calculates the view location of the image which 
should be displayed on the next moment, and the value of the direction of a look from the 
movement magnitude of the relative view which the user inputted as the view of an image on 
display, and the value of the direction of a look using the directions input means 108 by this time 
which the display image hysteresis storage means 109 holds, and the direction of a look. 
[0184] To the view location and the direction of a look which the image location count means 
110 determined, the image reading means 1505 determines which division image is read, and is 
read from the view positional information of the division image which the division image storage 
means 1 504 holds, and image positional information. 

[0185] The image composition means 1506 compounds the image of the direction of a look which 
the image location count means 110 determined from the image data which the image reading 
means 1505 read. 

[01 86] The display means 113 displays the image which the image composition means 1 507 
compounded Drawing 16 shows the outline of perimeter photography equipment 1501, the image 
which it photos, and a division image. In drawing 16 . the field where in a fish-eye lens and 1 602 
a camera and 1 603 become the image side of a camera and 1 604 becomes [ 203 / a truck and 
1 601 ] a candidate for photography, the image which photoed 1 605 with perimeter photography 
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equipment 1501. the field to which, as for 1606. an image is reflected, and 1607 are division 
images. 

[01 87] A fish-eye lens 1 601 covers the field 1604 of 90 degrees oi^ more vertically and 
horizontally from a core, respectively, and is attached in a camera 1602. A camera 1602 is 
attached in a truck 203 at vertical facing up. It lets a fish-eye lens 1601 pass, and a surrounding 
sight carries out image formation to the image side 1 603, By having such structure, the sight of 
the direction of the perimeter of one certain view can be stored in the image of one sheet By 
taking a photograph continuously, moving a truck 203. the perimeter image of the view when it 
continues in space is photoed. 

[01 88] If a photograph is taken with perimeter photography equipment 1 501 . an image as shown 
in an image 1605 will be obtained. The sight of the perimeter which actually let the fish-eye lens 
1601 pass to the field 1 606 on this image 1 605 is copied. The direction to which the travelling 
direction of perimeter photography equipment 1501 is reflected is used as the reference axis of 
an image. When the coordinate of the pixel on an image is expressed with the distance phi from 
this reference axis, the angle theta to make, and the image core O, Pixel p (theta, phi) is mapped 
on the straight line of a direction (theta. phi') in the space centering on a view. Here, by 
distortion by the fish-eye lens, although the relation of phi and phi' is nonlinear, it can ask for the 
multiplier of the high order polynomial which constitutes transformation by performing a 
calibration beforehand. In this way. since the pixel on a photography image is matched with the 
direction of a look in the space centering on a view, the image which exists in the location of the 
arbitration in space is compoundable from a photography image. 

[01 89] As shown in drawing 16 -c. the image division means 1 502 carries out the division-into- 
equal-parts rate of the image formation field in the image which perimeter photography 
equipment 1501 photoed in the direction of theta, and generates the division image 1607 of n 
sheets. 

[01 90] The division image location means 1 503 sets up the range of the bearing of the exposure 
axis theta and phi which the image covers to each division image. 

[0191] The image composition means 1506 sets up a virtual image in the space centering on a 
view, and asks for the pixel on the division image corresponding to the pixel on this virtual image 
by the above-mentioned transformation. When there is no pixel on the coordinate searched for. 
it approximates using the information on a neighboring pixel. The value about all the pixels on a 
virtual image is calculated, and an image is compounded. 

[0192] The image as for which image composition which was described above, and a display 
changed the view location and the direction of a look as the wish of a user continuously 
repeatedly according to directions of a user is reproduced. 

[0193] Since the image of a view location and the direction of a look according to directions of a 
user can be continuously compounded and displayed by performing the above processings also 
to the image data photoed while moving the perimeter photography equipment which stores the 
perimeter in the image of one sheet as mentioned above according to this example, the presence 
of an image can be raised. 

[0194] Moreover, since only data required at the time of composition can be read by dividing and 
holding a photography image, image reconstruction can be performed at a high speed. 
[01 95] Moreover, the image quality of a playback image can be raised by compounding a direct 
presentation image from the divided photography image. 

[01 96] (Example 7) Drawing 1 7 shows the configuration of the seventh example realized using 
the 14th means of this invention. 

[01 97] In drawing 17 . perimeter photography equipment has structure as shown in drawing 2 . 
divides the perimeter image of a certain view in the direction of plurality, and can photo 101 by 
once. This photography equipment is fixed to one place, fixed time amount continuation is carried 
out and the sight around it is photoed. 

[01 98] The A/D-conversion means 1 02 digital-data-izes the image photoed in this way. The 
image amendment means 103 amends dispersion in the brightness by the property of a camera 
etc. to the digitized image. The image storage means 1 04 accumulates the image data of the 
image which the image amendment means 1 03 amended. 
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[0199] The panorama image-sequences composition means 105 performs generating the 
panorama image in the time of day in all photography time of day using the image data which the 
image storage means 104 holds from the image of the direction of plurality photoed at the same 
time of day. In this way. a time series panorama image is generated. 

[0200] The panorama image storage means 1 701 accumulates the image data of the time series 
panorama image made in this way. 

[0201] The directions input means 1702 shows the relative migration direction of a look, while a 
user looks at a display image. As a migration direction of a look, rotation of the four directions of 
the right-and-left upper and lower sides can be specified. 

[0202] The hysteresis information on the direction of a look of the image which the display image 
hysteresis storage means 1703 is based on directions of a user, and is compounded and 
displayed serially is held. 

[0203] The image location count means 1 704 calculates the value of the direction of a look of 
the image displayed on a degree from the value of the direction of a look of the display image in 
this time which the display image hysteresis storage means 1 703 holds, and the movement 
magnitude of the direction of a look which the user inputted using the directions input means 
1702. 

[0204] The image reading means 1705 reads the image data of the panorama image in the next 
time of day of the panorama image which compounded the display image in this time from the 
panorama image storage means 1 701 . 

[0205] The image composition means 112 is compounded by performing the perspective 
projection [ image / for which the image location count means 1704 asked / of the direction of a 
look ] centering on a view from the image data of the panorama image which the image reading 
means 1 705 read 

[0206] The display means 113 displays the image compounded in this way. A user issues 
directions again to this display image, and image location count to it. image composition, and 
image display are performed again. It repeats until it gives directions of such a user and 
directions of a user's termination of composition of the image to it, and a display. 
[0207] Since an impression as if the event which occurred around the view at the time of 
photography by photoing time amount later on, compounding the image of the direction of a look 
of arbitration which a user desires at the time of playback, and displaying ail the directions 
around one point based on the photography image had happened at the time of playback can be 
given as mentioned above according to this example, the presence of an image can be raised. 
[0208] In addition, although it is a thing using a plane mirror as equipment which photos the 
direction of the perimeter and being explained by this example, the same effectiveness is 
acquired even if it uses the time series image which the field angle photoed using the large fish- 
eye lens. 

[0209] (Example 8) Drawing 18 shows the configuration of the image storage in the eighth 
example realized using the 15th means of this invention. 

[0210] For a head control section and 1803. as for a head and 1805. in drawing 18 . a disk 
control section and 1 804 are [ 1 801 / a controller and 1 802 / a magnetic disk and 1 806 ] the 

signal-processing sections. 

[0211] A controller 1801 controls R/W of the data on a disk. Based on directions of a controller 
1801. the head control section 1802 moves a head 1804 to a predetermined location, and the 
disk control section 1 803 rotates a magnetic disk 1 805. 

[0212] A head 1 804 writes in the image data changed into the analog signal by the signal- 
processing section 1 806 on a magnetic disk 1 805. 

[0213] Moreover, the analog signal which a head 1804 reads in on a magnetic disk 1805 is 
changed into digital data by the signal-processing section 1806. 

[0214] The magnetic disk 1805 used as a record medium of image data has a periphery-like 
recording track, and data are recorded on the truck. 

[0215] Drawing 1 9 shows the outline of the data arrangement on a magnetic disk 1805. Here, 
paying attention to View n and the image in direction of look thetai, the image data is set with I 
(n. i). The direction of a look is equal respectively and the image data I (n-1. i) and I (n+1, i) of 
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the image with which the view location adjoined is recorded on the part before and behind the 
truck with which this image data I (n. i) is recorded. Moreover, a view location is equal and the 
image data I (n, 1-1) and I (n, i+1) of the image with which the direction of a look adjoined is 
recorded on the same location of an adjoining truck. Thus, it performs putting in order the view 
location contiguous to the record section which continues or adjoins, and image data with the 
direction of a look to all image data on a record medium. 

[0216] Or the direction of a look would not adjoin in the view location where the image 
considered that image data I (n, i) is read, will read to a degree according to directions of a user 
supposing the user is shown the image compounded based on it, and a demand is performed now 
is the same, the direction of a look is the same and a view location adjoins. If the above data are 
arranged, since it is in the field to which all the required image data approached, even if the 
image of which direction is specified, it can read immediately. 

[021 7] As mentioned above, according to this example, by arranging the image data of the image 
with which a view location and the direction of a look continue on the recording track with which 
a disk-like record medium continues or adjoins, since the image data as which read-out was 
specified can be read immediately, image reconstruction can be performed at a high speed. 
[0218] In addition, in this example, although the case where a magnetic disk was used as a 
record medium was described, if it is a record medium with a disk [, such as an optical disk and a 
magneto-optic disk, ]-like configuration as a record medium, the same effectiveness will be 
acquired by arranging data which were described above. 

[0219] (Example 9) Drawing 20 shows the configuration of the image storage in the ninth example 
realized using the 16th means and 17th means of this invention. 

[0220] In drawing 20 , the magnetic disk with which 2001 consists of a controller and a disk of 
plurality [ 2002 / 2005 / a head control section the head of plurality / 2003 / 2004 / a disk 

control section and /, and ], and 2006 are the signal-processing sections. 

[0221] A controller 2001 controls R/W of the data on two or more disks. Based on directions of 
a controller 2001 , the disk control section 2003 rotates a magnetic disk 2005, and the head 
control section 2002 moves the head 2004 corresponding to each disk to a predetermined 
location. 

[0222] A head 2004 writes in the image data changed into the analog signal by the signal- 
processing section 2006 on a magnetic disk 2005. 

[0223] Moreover, the analog signal which a head 2004 reads in on a magnetic disk 2005 is 
changed into digital data by the signal-processing section 2006. 

[0224] Drawing 21 shows the outline of arrangement of the image data on a magnetic disk 2005. 
[0225] To the image sequences which put the perimeter image in a continuous view in order, the 
perimeter of a view is divided in the direction of k pieces, k image sequences are made, and the 
image data is recorded on k disks, respectively. 

[0226] At this time, the image data of the image in the same view is arranged on the same truck 
on each disk. Moreover, the image of a continuous view is arranged on a continuous truck. 
[0227] If read-out of image data is directed at the time of image reconstruction, a head will be 
moved to the truck which stores the image data directed on the disk, and it will read 
[0228] Although the number of the disks with which the image data which should be read is 
stored at this time is one. a head is moved to the truck with which the image data of the image 
of the same view location as the image to read is stored also to all other disks. 
[0229] That is, when read-out of the image data of View n and the direction i of a look is 
specified, in all disks, a head is moved to the truck with which the image data of the image in 
View n was stored. 

[0230] In this way, since the head on the i+1 st disks is moving to the truck with which the image 
of View n is already stored when the image with which read-out is directed next will be an image 
of View n and the direction i+1 of a look, if the head is moved, image data can be read 
immediately. 

[0231] Furthermore, since the image of the same view is recorded on the same truck, the 

movement magnitude of a head is the same in all disks, and control becomes easy. 

[0232] By arranging the division image data of the same view in the same location on two or 
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more are recording media according to this example, as described above, and interlocking all 
heads at the time of image read-out, and making it move, since time amount required for read- 
out of image data when the direction of a look is changed becomes small, image reconstruction 
can be performed at a high speed. 

[0233] In addition, in this example, although the case where a magnetic disk was used as a 
record medium was described, if it is a record medium with a disk [, such as an optical disk and a 
magneto-optic disk, ]-like configuration as a record medium, the same effectiveness will be 
acquired by arranging data which were described above. 

[0234] (Example 10) Drawing 22 shows the configuration of the tenth example realized using the 
18th means of this invention. 

[0235] For 2201 , as for the maximum data size detecting element and 2203, in drawing 22 , the 
picture compression section and 2202 are [ the data supplement section and 2204 ] archive 
media. 

[0236] The picture compression section 2201 compresses the image data of an image which 
divided the perimeter image around a view in two or more directions by removing redundancy. 
Here, the data size after compression of Image I is set with Si. Since the data size after 
compression changes with contents of each image, as for the maximum data size detecting 
element 2201, an image with the largest data size is detected out of all compression images. The 
data size of the detected compression image is set with Smax. The data supplement section 
2203 adds the dummy data which has the magnitude of Smax-Si in the last of the image data to 
each compression image. In this way, the image data size of all compression images is unified 
into S. In this way, the generated image data is recorded on an archive medium 2204. 
[0237] Drawing 23 is the outline of the data arrangement by the archive medium 2204. A 
magnetic disk is used as an archive medium. The image data which added dummy data to the 
recording track on a magnetic disk is recorded. At this time, the image data of the image of an 
adjoining view and the direction of a look is recorded on the truck which continues, respectively 
or adjoins. Thus, if it records, since the image size of each image is unified, the record location of 
the image data on a truck is called for easily. 

[0238] As mentioned above, by this example, since it can ask for the record location of data 
easily by adding dummy data to the image data after compression, unifying data size and 
recording, the time amount concerning loading of an image becomes small, and image 
reconstruction can be performed at a high speed. 

[0239] In addition, in this example, although the case where a magnetic disk was used as a 
record medium was described, if it is a record medium with a disk [, such as an optical disk and a 
magneto-optic disk, ]-like configuration as a record medium, the same effectiveness will be 
acquired by arranging data which were described above. 
[0240] 

[Effect of the Invention] As mentioned above, while compounding the panorama image centering 
on a view beforehand to the image data of the image which photoed the perimeter by making into 
a view the point which draws the locus of arbitration and moves in the inside of space according 
to this invention The panorama image equipment which provides a user with the high image of 
presence is realizable by repeating compounding the image of a view location and the direction of 
a look according to directions of a user from a panorama image, and displaying it at the time of 
playback. 

[0241] Moreover, the panorama image equipment which offers a high definition image is realizable 
by compounding a direct presentation image from a photography image at the time of image 
composition. 

[0242] Moreover, since time amount which read-out of image data takes by dividing and treating 
an image can be made small, the panorama image equipment which performs high-speed image 
reconstruction is realizable: 

[0243] Moreover, since time amount which read-out of image data takes by controlling two or 
more read-out heads can be made small while arranging image data the optimal on an are 
recording medium, the panorama image equipment which performs high-speed image 
reconstruction is realizable. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] The block diagram of the panorama image equipment in the first example 
[Drawing 2] The block diagram of the perimeter photography equipment in the first example 
[Drawing 3] The physical relationship Fig. of a photography image and a panorama image in the 
first example 

[Drawing 4] The flow chart of the processing at the time of the image reconstruction in the first 
example 

[Drawing 5] The block diagram of the panorama image equipment in the second example 

[Drawing 6] The block diagram of the perimeter photography equipment in the second example 

[Drawing 7] The block diagram of the panorama image equipment in the third example 

[Drawing 8] The conceptual diagram of the image division in the third example 

[Drawing 9] The flow chart of the processing at the time of the image reconstruction in the third 

example 

[Drawing 10] Drawing showing the regulation for the image location anticipation in the third 
example 

[Drawing 1 1] The block diagram of the panorama image equipment in the fourth example 
[Drawing 12] The physical relationship Fig. of a photography image and a virtual image in the 
fourth example 

[Drawing 13] The block diagram of the panorama image equipment in the fifth example 
[Drawing 14] The image physical relationship Fig. at the time of the image composition in the 
fifth example 

[Drawing 15] The block diagram of the panorama image equipment in the sixth example 
[Drawing 16] The perimeter photography equipment in the sixth example, and the schematic 
diagram of a photography image 

[Drawing 17] The block diagram of the panorama image equipment in the seventh example 

[Drawing 18] The block diagram of the image storage in the eighth example 

[Drawing 19] The schematic diagram of the data arrangement in the eighth example 

[Drawing 20] The block diagram of the image storage in the ninth example 

[Drawing 21] The schematic diagram of the data arrangement in the ninth example 

[Drawing 22] It is a block diagram to the image storage in the tenth example. 

[Drawing 23] The schematic diagram of the data arrangement in the tenth example 

[Drawing 24] The block diagram of the panorama image equipment in the conventional example 

[Drawing 25] The physical relationship Fig. of a photography image and a panorama image in the 

conventional example 

[Description of Notations] 

101 Perimeter Photography Equipment 

1 02 A/D-Conversion Means 

103 Image Amendment Means 

1 04 Image Storage Means 

1 05 Panorama Image-Sequences Composition Means 

1 06 View Positional Information Setting Means 
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1 07 Panorama Image Storage Means 

108 Directions Input Means 

109 Display Image Hysteresis Storage Means 

1 10 Image Location Count Means 

1 1 1 Image Reading Means 

112 Image Composition Means 

1 1 3 Display Means 
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[Drawing 22] 
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[Drawing 14] 
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